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Behr, Merlyn J*; Waehsmuth, Ipke; Post f Thomas R*; and Leah, Richard. 
ORDER AND EQUIVALENCE OF RATIONAL NUMBERS: A CLINICAL TEACHING 
EXPERIMENT. Journal for Research in Mathematics Education 15: 323-341; 
November 1984. 



Abstract and comments prepared for I.M.E. by DONALD J. DESSART , The 
University of Tennessee at Knoxville. 

1 . Purpose 

This study investigated some of the understandings of the order 
and equivalence of rationale numbers by 12 fourth-grade students who 
were each interviewed 11 times during an 18-veek teaching experiment. 
In particular, the students 1 strategies for comparing three distinct 
types of fraction pairs were identified, 

2. Rationale 

Recent national asnessments have demonstrated that many children 
have great difficulty learning anJ applying concepts of rational 
numbers. Children, for example, add the numerators and denominators 
of two fractions when finding the sum of these fractions, e.g. , 

1 1 " !• Man y children do not consider the numerator and 

2 3 5 

denominator of a fraction in relation to one another but rather think 
of them as separate objects, thus giving 19 or 21 as an estimate for 

12 + 7. 

13 8 

Previous research has concentrated upon children's thinking about 
rational numbers in ordinary school situations, has investigated 
students' learning with experimental materials, or has concentrated 
upon children's handling of the equivalence of fractions in relation 
to proportional reasoning. These studies have usually used 
clinical-type research procedures. 
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This study is a report from the Rational Number Project which was 
supported by the National Science Foundation at several universities 
during 1979 through 1983. A major focus of this Project was the role 
of physical models in learning concepts about rational numbers. 

3. R esearch Design and Procedures 

An 18-week teaching experiment was conducted in schools of St. Paul, 
Minnesota and DeKalb, Illinois between October 1980 and March 1981. 
Twelve fourth-grade students were subjects of the study, with six at 
each site. The program of studies consisted of 13 lessons, identical 
at each location. The children worked individually or in groups but 
did not receive any other formal instruction on rational numbers. 

Part-whole interpretations of rational numbers were introduced by 
ircular and rectangular pieces of laminated construction paper. Each 
it fraction, 1, for n from 1 to 10, 12, and 15 was presented by 
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different colored paper. Later instruction utilized Cuisenaire rods, 
paperfolding, and poker chips. The complete instructional phase 
included five topics: naming fractions, equivalent fractions, 
comparing fractions, adding fractions of the same denominator, and 
multiplying fractions. 

Each of the 12 children was interviewed 11 times or about every 
8 days during the 18-week period. The interviews were audio- or 
video-taped for later transcription and analysis. The interviews 
ranged across the major strands of the Project. However, this paper 
reported only those observations dealing with the order and equivalence 
of rational numbers, which came up during most of the interviews. j 

I 
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4. Findings 



The major concern of the interviewers was to provide an explication 
of the thought processes of the children from which strategies for 
determining order and equivalence could be inferred. The strategies 
were reported according to three classes of fractions: (a) fractions 
with the same numerators, (b) fractions with the same denominators, 
and Cc) fractions with different numerators and denominators. 

In analyzing the interviews covering fractions with the same 
numerators, five distinct strategies were encountered: (a) Numerator 
and Denominator: a strategy in which the child decided that if the 
same numerator was present in both fractions, the smaller fraction 
had the larger denominator; (b) Denominator Only: a strategy in 
which the explanation offered by the child only referred to the 
denominator of the fraction; (c) Reference Point: a strategy in 
which the child referred both fractions to a third fraction; (d) 
Manipulative: the child explained his or her decision by using 
pictures or roanipulatives; and (e) Whole Number Dominance: this 
Invalid strategy centered entirely on the size of the denominator — the 
larger fraction possessed the larger denominator. At both Minneapolis 
and DeKalb the most prevalent strategy late during instruction was the 
Denominator Only Strategy; whereas, early during instruction the Whole 
Number Dominance Strategy was prevalent. 

In dealing with the second class of fractions (same denominators), 
the children used five strategies: Numerator and Denominator, 
Reference Point, Manipulative, Whole Number Consistent, and Incorrect 
Numerator and Denominator, The most prevalent strategy used at both 
DeKalb and Minneapolis was the Numerator and Denominator Strategy. 

m 

Six strategies were suggested by the children in dealing with 
fractions having different numerators and denominators. These were: 
Application of Ratios, Reference Point, Manipulative, Addition, 
Incomplete Proportion, and Whole Number Dominance, The most prevalent 
strategy at DeKalb was Application of Ratios and at Minneapolis was 
Manipulative. 
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5. Interpretations 

The authors provide the following implications of their work: 

a. Most children after having been provided adequate instruction 
deal with questions of order and equivalence of fractions by the end 
ot" the fourth grade. 

b. Understanding order and equivalence depends upon an under- 
standing of a compensatory relation among the size and the number of 
equal parts in a partitioned unit. 

c. More time should be devoted to Instruction in this compensatory 
relation than is provided in most modern curricula. 

d. Children who are insecure in their understandings of rational 
number concepts often display an interference from their previous 
understandings of whole number concepts. 

e. Some children invented the Reference Point Strategy which may 
be an outgrowth of or related to estimation skills. 

f. Some children remain dependent upon manipulatlves to the point 
of being inefficient. 

Abstractor's Comments 

Investigations of this kind often raise numerous questions. For 

example: 

1. The number of children selected at each site was very small 
(6). Were they randomly selected? Were some criteria used in their 

selection? Were the children bright, average, slow? These items 
certainly would affect the observed strategies. 
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2. What was the role of the classroom teachers? Were they 
specially trained for this experiment? Were they rated as good, 
average, or poor teachers? 

3. The prevalence of certain strategies at both sites, e.g., 
"Denominator Onl;, " with Unit Fractions, might have reflected teachers' 
attempts to instill (drill) some "rule behavior" in the children. 

Was this done? 

A. It appears that the distinction between the "Numerator and 
Denominator" and the "Denominator Only" strategies with unit fractions 
Is very slight. It seems the second strategy can only instill "bad 
habits" in children! 

Overall, the investigators are to be congratulated on an excellent 
piece of work that should add significantly to the research literature 
on childrens* learning of rational number concepts* It is a pity that 
relatively few elementary school teachers will read the report in 
JRMF, or this review! The problem of making available "ideas" from 
research and not necessarily stable, definitive results (Do they ever 
exist?) to school practitioners is a serious problem which mathematics 
educators should soon face if the research enterprise is tQ^ever 
significantly aid in the improvement of instruction in the schools. 
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Kulkerson, Katherine F. ; Calassi, John P.; and Calassi, Merna Dee. 
RKI.ATTON BETWEEN COGNITIONS AND PERFORMANCE IN MATH ANXIOUS STUDENTS : 
A FAILURE OF COGNITIVE THEORY? Journal of Counseli ng Psychology 
31 : 376-382; July 1984. 



Abstract and comments prepared for I.M.E. by GRACE M. BURTON, 
University of North Carolina, Wilmington, North Carclina. 



1 . Purpose 

This investigation was de.-.igned to test three specific questions: 

(a) How do cognitions during mathematical problem solving vary as 
.i t unction of mathematics anxiety level? 

(b) How do cognitions of men and women differ when they are 
solving mathematical problems? 

(o) To what extent is variation in mathematics performance 
accounted for by variation in cognitions? 



? - R^tJunaJ_e 

Mathematics anxiety, sex differences in mathematical achievement 
and attitude, and the role of "self-talk" in mathematics achievement 
have been topics of much concern in recent years. Although a person's 
self-critical and self-congratulatory tendencies have been considered 
important mediators of behavior and behavior change, empirical 
evidence for this claim is scanty. In addition, it is not known if 
differences between "self-talk" patterns of men and women or between 
math-anxious and non-math-anxious students exist. Delineation of 
these differences is important if counseling math anxious students is 
to be maximally effective. 
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5 . Res varch Pes inn and Procedures 

During the summer of 1981, five hundred eighty-two students from 
hit roduetorv i nurses at a large state university were given the 
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■Mathematics Anxiety Rating Scale (MARS), a five-point Likert-type 
scale of 98 items. From those scoring in the bottom one-third and 
from those scoring in the tpp one-third of the MARS distribution, a 
random seltrtion, stratified by sex, was chosen. The sample consisted 
of a high-anxious group of 16 men and 18 women and a low-anxious group 
of 19 men "and 18 women. The mean on the MARS for the high-anxious 
men was 229.38; that of , the high-anxious women 254.28. The mean on 
the MARS for the low-anxious men was 126.63; that of the low-anxious 
women was 133.78. SAT-Q scores were obtained for each subgroup. Those 
fof the high- and low-anxious men were 494.38 and 591.58, respectively; 
those for the high- and low-anxious women were 480.56 and 526.11, 
respectively. 

Following a practice session, the students were asked to say 
aloud whatever thoughts they had as they completed six mathematics 
problems drawn from the SAT. They then were asked to complete silently 
a set of 12 problems and the Test Anxiety Inventory. The student 
verbalizations were independently coded by two trained researchers 
blind to the hypothesis of the study. The coding categories used 
were: review of information, strategic plans, conclusions and 
solutions, attention control, self-facilitation, irrelevancies , 
self-inhibition, neutral statements, silence, and unclassif iable 
statements. A percentage of agreement close to 95% was obtained. 

Both, multivariate analyses and univariate analyses were completed. 
In the regression using cognition data, sex was entered first, 
followed by math anxiety, then a sex x anxiety interaction. In that 
of performance data, n the -cognition categories of attention control, 
self-facilitation, irrelevancies, and self-inhibition were entered 
together first, and then the other seven categories were entered next. 
Finally, a multiple regression analysis was conducted using all 11 
categories as predictor variables. 
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4. Find lags 

In t MAN OVA, there were no significant differences (p>.05) as a 
function of sex, mathematics anxiety or sex by anxiety interaction on 
the 11 cognitive variables. On the combined problem set and on^the 
problem sets separately, while neither sex nor anxiety-sex interaction 
were significant, anxiety approached significance (.05 < p < .10) . 

When the cognitive variables were subjected to univariate analysis, 
there were only three significant effects disclosed: men reported 
more irrelevancies and more neutral statements than did women, and ' 
women used more strategic calculations than did men. No significant 
differences were found in performance due to sex, anxiety or their 
interaction, although the latter approached significance. 

In the multivariate regression analysis using performance on the 
combined problems, the second set of cognitive variables accounted 
for a significant (p<.05) amount of variance (20.2%) beyond that of 
the first set of cognitive variables. On the think-aloud problems, 
neither set was significant. On the silent problems, however, both 
sets accounted for a significant (p <• 05) part- of the variance, with 
the first set accounting for 14.7% and the second an additional 23.0%. , 

5. I n t e rp r et_a t ions 

That cognitions as students solve mathematics problems do not vary 
as a function of math anxiety, sex, or their interaction may point to 
a general set towards mathematics rather than a particular set towards 
the specific mathematics task of problem solving. It might also be 
that the saliency of the cognitions, not merely thei: number, is 
important . 

Because < ognitions were more important in the silent situation, it 
may be that thinking aloud does impair performance, contrary to the 
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current literature oh this topic. Because of the lack of a strong 
data base on the basic assumption underlying cognitive theory, much 
research is needed before effective counseling strategies can be 

i 

devised. 

r 

Ab s tractor 1 s Comment s 

The ar^a investigated in this study is an important one as data 
on the self-talk of students as they engage in mathematical tasks is 
scarce. Such data might h~lp counselors and teachers of mathematics 
prcvide effective and efficient assistance to students experiencing 
a less-than-maximal level of mathematical f unctiSning, whether they 
are math-anxious or not. Although I acknowledge the importance of 
the area, I have reservations as to the value of this study on three 
counts — the adequacy of the bibliography, the nature of the data, and 
the significance of the findings. . m 

The fact that this article is essentially a report of a doctoral 
dissertation completed before October 1983 may account for the age of 
many of the bibliographic entries. It does not account for the 
paucity of references to related areas such as mathematics education. 
Mathematics anxiety and sex differences have been the focus of much 
attention in the last few years; authors of a paper which purports 
to deal with these issues should, it seems to me, assure their readers 
of their familiarity with recent studies and summaries of research 
relating to the topics under discussion. 

With respect to the study itself, it appears one must trust, as 
the authors evidently did, that what one sa ys one is thinking is 
indeed what is in one f s mind, I am not convinced this is the case. 
Additionally, the chosen categories do not appear to be distinct* 
Although the raters apparently experienced little difficulty on this 
score, there are many statements I would have difficulty assigning to 
*>ne and only one category. I would not know, for example, whether to 

x 
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code "I've got it!" as Inferring a solution (conclusion- and solutions) 
or as positively evaluating knowledge (self -facilitation) . "I've seen 
one like this before" could, it seems to me, be a neutral, an 
uixclassifiable, or even, again, a self-f acilitative statement. 

Finally, the g^hered data appears to have been subjected to 
extremely detailed treatment, given the size of the sample (n - 71). 
Despite this, few significant differences were found due to sex, 
anxiety or the interaction of sex with anxiety and those that were 
found may be less than useful. It Is hard to know what to do with 
the finding, for example, that men used more irrelevancies and more 
neutral statements than women did. 

I can whole-heartedly concur with the authors that ;, ccgnitlve 
and cognitive-behavioral approaches to counseling are in the early 
stages of development" (p. 381). I believe it is an Important area 
to research and I hope that further investigation will draw on the 
literature of the many fields that impact on this interdisciplinary 
area and that they will uncover information both statistically and 
clinically significant. 
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Hannafin, Michael J. FRUITS AND FALLACIES ON INSTRUCTIONAL SYSTEMS: 
EFFECTS OF AN INSTRUCTIONAL SYSTEMS APPROACH ON THE CONCEPT ATTAINMENT 
OF ANGLO AND HISPANIC STUDENTS. American Educational Research Journal 
20: 237-249; Summer 1983. 

Abstract and comments prepared for I.M.E. by jON M. ENGELHARDT, Arizona 
State University. 

1. Purpose 

The study purported to compare traditional instruction and an 
"instructional system" approach to mathematics concept attainment 
across factors of gender and ethnicity. Concept attainment was defined 
as relative mastery on three types of mathematics skills (those 
previously taught, those not previously taught but for which 
prerequisites have been taught, and those not previously taught and 
for which previously taught material was not prerequisite). 

2. Research Design and Procedures 

For the sixth-grade sample in this study (four classrooms, suburban 
middle-class school district, 49 Anglos and 36 Hispanic students, 
English primary language), these three types of skills (respectively) 
were whole number computations, adding/subtracting decimals, and 
adding/subtracting fractions. 

Traditional instruction was defined in terms of the materials 
use d — sixth-grade district-adopted mathematics text, a scope-and- 
sequence chart of objectives, text-related commercial practice and 
drill materials, and district-designed supplementary exercises. The 
instructional system was defined to include the above materials plus 
individual and class profile sheets, a pretest, and criterion-referenced 
skill quizzes; it also included a hierarchically sequenced set of 
computational skills ordinarily taught from grade 2 through 8 and 
realtively standardized procedures of test-teach (based on test)-test. 
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There was no indication of the nature of the instruction in either 
the traditional or instructional system approach. The experiment 
extended ovtr an eight-month period. 

3. Findings 

It was the thesis of the study that systematic instruction would 
nullify the supposed influence of gender and ethnicity, The results 
did, in fact, offer some support for this. Where previous instruction 
was related to current instruction, differences across ethnic groups 
(although not type of instruction) were apparent, as expected; but 
where previous instruction was unrelated to current instruction, 
effects of ethnicity were greatly reduced and the instructional system 
approach was generally superior. No gender differences were found. 

4. Interpretations 

The researcher concluded that an instructional systems approach 
may "help to offset performance gaps often attributed to ethnic group" 
that presumably are facilitated (or at least not mitigated) by 
traditional instruction. 

Abstractor's Comments 

This investigation represents an important step into examining 
ethnicity (and gender) in relation to instructional variables, rather 
than simply looking at achievement. Furthermore, it attempted to look 
at instruction longitudinally (8 months) and more holistically 
(traditional versus systems approach). 

Although mathematics is used as a content area, the thrust of this 
study, however, is more general than specific to mathematics education. 
Unfortunately, despite its positive directions, mathematics educators 
will find the study problematic ferom a couple of different angles. 
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FirstV^ and a flaw common to many treatment comparison studies, is the 
lack ot specif icity in describing the instructional treatments. Not 
only is the information so sketchy as to defy understanding of the 
general naturae of the treatments — traditional versus systems — but 
the nature of instruction on whole numbers, decimals, and fractions 
across the two treatments is indeterminable. Mathematics educators 
examining this study will be greatly concerned by the lack of 
specificity about instructional treatments, both generally and 
specifically. There may have been systematic differences in 
instruction between the two treatments other than those proposed that 
offer alternate explanations for observed differences in effect. 

A second area of concern for many mathematics educators will be 
the operational definition of concept attainment, i.e., mastery of 
various computational skills. Without going headlong into a 
discussion of concept learning, mathematics educators would generally 
see the learning tasks of this study as procedure learning. Granted 
there may well be concept learning involved in learning these 
procedures, but from the absence of information on instructional 
treatments, either or bot^T types of instruction may have stressed the 
mechanical steps to computation without reference to the mathematical 
structure or basic mathematical concepts out of which they arise. 
Furthermore, it is likely that a number of relevant mathematical 
concepts were attained by students regardless of their performance on 
a measure of computational skill. 

This investigation seems to raise more questions for the 
mathematics educator than it answers. Its results are encouraging in 
some measure, but a great deal more information is necessary before 
it will influence future research or begin to have impact on 
educational practice. 
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Larson, Carol Novillis and Slaughter, Helen. THE USE OF MAN IPUUT IVES 
AND GAMES IN SELECTED ELEMENTARY SCHOOL CLASSROOMS, FROM AN 
ETHNOGRAPHIC STUDY, Focus on Learning Problems In Mathematics 
6: 31-49; Winter and Spring 1984. 

Abstract and comments prepared for I.M.E. by LOYE Y. "MICKEY" HOLLIS, 
University of Houston-University Park. 



1. Purpose ! 

The study describes how manipulative s and games were used in nine 
elementary school classrooms over an extended period of time. 

2 . Rationale 

Detailed description of the classroom setting has been sorely 
lacking. Some of the areas related to the teaching of mathematics in 
which not enough is known are: (1) the amount of time the teachers 
allocate to mathematics instruction and the manner in which this time 
is used; (2) the extent to which teachers differentiate instruction; 
(3) the extent and nature of teachers' use of manipulative materials; 
and (4) the identification of the factors that determine teachers' use 
of non-text learning materials. 

1. Rese arch Desi g n and Procedures 

This ethnographic study used a sample of nine third- and fifth- 
grade teachers in eight sites. They were chosen from 27 Title I 
teachers who volunteered to receive classroom services from the 
mathematics project specialist. Target students who were low achievers 
in mathematics were identified in each classroom. 



Ten to 15 ethnographic observations were conducted in each 
classroom over a period of approximately five weeks by four trained 
assistants. Observations were scheduled before, during, and following 
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the delivery of classroom services by a mathematics project 
specialist . After each observation period of approximately one hour, 
the ethnographic assistants wrote narrative accounts (protocols) of 
the classroom events from their field notes. All data were reviewed 
by teacher collaborators during a two-day research colloquium at the 
end of the school year. 

4. Findings 

The .reported findings are in three categories: description of 
classrooms, manipulatives and games. 

Classrooms 

Three different organizational patterns (with some variation) 
were observed in the first week's observation prior to the time when 
the mathematics project specialist began. The organizational patterns 
w?re: (1) whole-group instruction with "extra help" for children 
having problems; (2) some type of small-group instruction; and (3) 
individual contracts. 

The organizational pattern changed in three of the classrooms when 
the mathematics program specialist was teaching the classroom. This 
change was to small-group instruction. 

Manipulatives 

The study provided a number of insights into the contribution of 
manipulatives to the nu...aematics learning of low-achieving students. 
Often low achievers were more successful doing manipulative activities 
than other mathematics activities. 
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There was a lack of discussion between teachers and students 
relating the mathematical symbols and equations to the manipulatives. 
In some cases after teachers used manipulatives for two or more weeks 
in a unit, they were surprised and disappointed that students were 
not successful on traditional abstract tasks. 



Games 



Children were observed playing mathematical games as a regular 
part of the instructional program in six of- the research classrooms. 
Two types of games were described in the protocols: games that 
provided practice of previous instruction and games that were used 
as vehicles for instruction to help children develop new 
mathematical concepts. 

Management of games appeared to be an important factor in the 
successful use of games in the classroom. Game groups were more 
easily managed when more than one adult was working in the classroom 
and an adult led the game groups. Some of the problems that occurred 
in single teacher rooms were: (1) some games did not hold students' 
interest for the period of time required for the teacher's 
instructional group and the independent group to finish their 
activities; and (2) the social demands of the game activity, such as 
conflict resolution, abiding by rules, fair treatment of others, and 
negotiating first turn, dominated %he groups' behavior. 

5 . Inter pretations 

Two main problems were evident: the difficulty low-achieving 

children had with transfer from the concrete to the abstract level 

and the management of peer game groups. Both of these problems seem 

to be related to a lack of individualized instruction. 
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The small-group, organizational patterns were chosen by the 
teachers not for the purpose of varying instruction, but so that 
they could more easily use the manipulatives and games. Despite this, 
instruction in small groups especially benefited the lower achievers, 
who participated more actively in the smaller groups than in the 
whole-group instruction. 

The mere use of manipulative materials in the classroom does not 
guarantee that mathematical learning is taking place. It would seem 
that in order for manipulatives to be effective teaching tools, the 
teacher must have (1) a good understanding of the relationship of 
models to mathematics, and (2) a good understanding of how to promote 
students' thinking about what they are doing with the models and how 
the models relate to mathematics. 

Some suggestions for using games are: 

1. Games should be chosen that reinforce present instruction. 

2. Consideration must be given to the complexity of the game. 

3. When selecting games, teachers should try to foresee problems 
such as children ''s disruptive behavior. 

4. An answer key of some type is required if children are to play 
games without an adult present in the group. 

5. An adult should play at least one round of a new game with 
the children. 

6. The game group is a good spot for a teacher aide or a parent 
helper. 

Abs tractor ' s Commen t s 

This report should be of interest to mathematics educators, 
mathematics supervisors, and teachers of mathematics. The report is 
informative and identifies some critical aspects of using manipulatives 
and games. They are similar or identical to some of my own 
observations. When a study supports your own beliefs, you do have a 
tendency to view it positively. Such is the case with this report. 
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The report describes enough of the ethnographic procedures 
to give the reader some insights into the procedure. More studies 
of this type are needed. I would hope this one would encourage 
others. 
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Minato, Saburoh. SOME MATHEMATICAL ATT1TUDINAL DATA ON EIGHTH GRADE 
STUDENTS IN JAPAN MEASURED BY A SEMANTIC DIFFERENTIAL. Educational 
Studies In Mathematics 14: 19-38; February 1983. 

Abstract and comments prepared for I.M.E. by LEWIS R. AIKEN, Pepperdine 
University, Malibu. 

1 . Purpose 

The purpose of this study was to construct and validate a semantic 
differential scale of attitudes toward mathematics for Japanese school 
children, and to provide additional statistical data on sex-related 
differences in attitudes toward mathematics. t 

2. Rationale 

Likert-type, Thurstone-type, and semantic differential scaled of 
attitude toward mathematics developed in English are briefly described. 
The author maintains, however, that existing semantic differential 
scales of attitude toward mathematics are not culture-free, thus 
justifying the construction and validation of an instrument of this 
kind in Japan. 

3. Research Design and Procedures 

The author's working definition of attitude is: "Attitude is a 
learned implicit process which is potentially bipolar, varies in its 
intensity, and is part of the internal mediational activity that 
operates between a stimulus and the individual's more overt evaluative 
response pattern." Using this definition as a starting point, the 
author proceeded to develop a semantic differential scale of attitude 
toward mathematics. The instrument, referred to as MSD, requires the 
respondent to rate the concept SCHOOL MATHEMATICS on 17 seven-point 
bipolar adjectival scales. Scores on each scale range from -3 
(extremely unfavorable) to +3 (extremely favorable). Total score on 
the MSD is the sum of scores on the 17 scales. 
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Participants in the study were 175 eighth-grade students (87 boys 
and 88 girls) in a public lower secondary school in a small Japanese 
city, Akita Prefecture. In sedition to the MSD, two other mathematics 
attitude instruments, Japanese translations of the Dutton Attitude 
Scale and another semantic differential scale devised by Ant ton en, were 
administered. Scores on a mathematics achievement test measuring 
computation with the four fundamental operations were also analyzed. 
The attitude instruments were administered at the beginning of the 
school year (April) to students in five classes containing 
approximately equal numbers of boys and girls. Each class was taught 
by one of two experienced women teachers licensed to teach mathematics 
in Japan. 

4. Findings 

An analysis of the ability of the 17 MSD items to discriminate 
between high and low scorers on the MSD instrument as a whole showed 
that every item possessed significant discrimination ability. An 
alternate item analysis procedure devised by Brinton confirmed the 
findings of the first item discrimination analysis. 

With respect to the validity of the MSD, total scores on the 
instrument correlated .67 with the Dutton Attitude scale and .47 with 
the Anttonen semantic differential scale; both correlations are 
statistically significant. Correlations of MSD scores with teacher's 
estimates of students' attitudes were .51 for boys, .55 for girls, 
and .52 for both groups. Test-retest reliability coefficients of the 
MSD over an interval of ten months were .67 for girls, .61 for boys, 
and .64 for combined groups. 

A centroid factor analysis was performed on the MSD scores of a 
group of elementary school teachers in a previous study. Four factors 
labeled Evaluation, Potency, Beauty, and clarity, were obtained. The 
correlations between MSD scores and scores on the mathematics 
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achievement test were .50 for boys, .53 for girls, and .51 for both 
groups combined. The mean MSD score was significant!/ higher for girls 
than for boys, but the mean achievement test scores of the two groups 
were not significantly different. Comparisons of boys' with girls' 
scores on the MSD items revealed several item means in favor of girls, 
especially items having high loadings on the Evaluation factor. 



5. Interpretation 

The author of this study concluded from the findings that: (1) 
overall, the MSD is superior to translated American attitude scales 
as a measure of attitude toward mathematics in Japan; (2) the 
reliability of the MSD is lower than might be expected; (.i) the 
validity of the MSD is higher than that «f comparable American 
instruments; (A) girls reveal more positive attitudes than boys on the 
MSD. 



Abstractor's Comments 



The article is fairly well written, although a bit long for an 
Instrument-development study. The author also has a tendency to "jump 
around" from the current study of eighth-grade students to his previous 
studies of elementary teachers. 

The rather modest test-retest reliability of the MSD may be 
attributable to the age of the respondents, the relatively long (10 
months) test-retest period, and the brevity of the instrument. 
Internal consistency coefficients (Kuder-Richardson, Cronbach alpha) 
should also have been computed. 

The higher- than-typical (at least in the U.S.A.) correlations of 
the MSD with achievement tests scores may be a reflection of the fact 
that the achievement test was on arithmetic fundamentals and was 
administered before rather than after the MSD, Also, the reader is not 
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told how much feedback the Btudents received about their mathematics 
achievement scores or what the precise time intervals were. Finally, 
the fact that girls had more positive mathematics attitudes th*n boys 
is not particularly surprising at the eighth-prade* level. It may re 
that changing social conditions in Japan, and perhaps the U.S. as well, 
are causing girls to develop more positive attitudes, tow?-.' mathematics. 
Be that as it may, girls' -attitudes toward mathematics are usually less 
positive than those of boys in high school and college than in the 
earlier grades, at least in the United States. Finally, the results 
of this study reinforce the writer's belief that one must be cautious 
in making cross-cultural generalizations when dealing with educational 
and psychological matters. 
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Moyer, John C; Sowder, Larry; Threadgill-Sowder , Judith; and 
Moyer, Margaret B. STORY PROBLEM FORMATS: DRAWN VERSUS VERBAL 
VERSUS TELEGRAPHIC. Journal for Research In Mathematics Education ■ 
15: 342-351; November 1984. 

Abstract and comments prepared for I.M.E. by GLEN BLUME, University 
of Iowa. 

1. Purpose 

The purpose of this study was to compare children's performance 
on ptory problems in three contexts: drawn (picture labeled with 
brief phrases), verbal (full sentences), and telegraphic (reduced 
verbiage). The study also examined the relative difficulty of the 
three formats for children with b ^h or low reading ability. 

2. Rationa le 

The authors note that a reduction of working-memory overload can 
be hypothesized as support for the use of the telegraphic rather than 
the verbal format. However, previous research by the authors found 
that "the telegraphic format was not superior to the verbal one and 
that high-ability students did somewhat better on problems in the 
verbal format, perhaps due to the richer contextual cues provided by 
complete sentences. 

The authors' previous research also found performance on the 
drawn format to be superior to the verbal format at Grade 5. Four 
arguments are given suggesting superiority of the drawn format: the 
drawing can reduce reading-related memory overload, the drawn format 
may help with recall of similar situations and may encourage the 
construction of appropriate images, it may induce students to link the 
meanings of the telegraphic phrases with the drawing and thereby aid 
understanding of the problem, and it may help organize information in 
the problem. 
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3. Research Design and Procedures 

The sample consisted of 854 students from three cities. Eight 
classes chosen by administrators were used in each of Grades 3 
through 7. 

Three parallel forms of a 24-item test were developed tor each 
grade and randomly assigned to subjects., Each form contained eight 
drawn, eight telegraphic and eight verbal items that were balanced 
for number size, operations required, and the like. Problems were 
one-step, multi-step, and extraneous data story problems requiring 
operations on- whole numbers, fractions, and decimals, as appropriate 
for each grade. The eight items in each format required 12 
operations; one point was scored for each correct operation-operand 
pair disregarding correctness of computation. Reliability (Cronbach's 
alpha) ranged from .79 to .88 for the three format subtests at the 
five graces. 

The Syntactic Similarities subtest of the Test of Reading 
Comprehension was administered to measure reading ability. Subjects 
were classified as "high" or "low" based on whether they scored above 
or below their grade- level mean. 

The 24-item story problem test was administered in two 25-minute 
sessions on two days, with the reading test given just prior to the 
second session. 

4. Findings 

For each reading It /el within each grade the mean for the eight 
drawn problems was higher than that for either of the other two 
formats. Across the grades, means in the three formats for the low 
readers were approximately four to six out of a possible score of 12. 
For high readers the means were approximately seven to ten. 



30 



25 



ANOVA with format scores as repeated measures indicated 
significant format, grade, and reading level effects and a 
significant format -by-reading-level interaction. Simple format 
effects were significant for both reading levels, with means for the 
drawn format significantly higher than means for the other two 
formats, which did not differ significantly from each other. The 
grade effect seemed to be due to easier tests at Grades 4 and 5, and 
the reading level effect reflected better performance by the high 
readers. 

5, Interpretations 

This study replicated the authors' previous studies in that the 
drawn format was superior and no significant difference was found 
between the telegraphic "and verbal formats. However, unlike the 
previous study, the verbal format was not easier than the telegraphic 
for the high-ability students (high readers). The authors contend 
that the superiority of the high readers over the low readers for all 
grades and formats suggests that "some difficulty in story problem 
solving may be due to reading-related memory overload" (p. 349). 

Low readers appear to profit more than high readers from the 
drawn format. The authors suggest that, although the drawn format 
helps both good and poor readers by making semantic processing 
easier, it helps poor readers even further by relieving memory 
overload. 

The authors question that use of the telegraphic format and 
suggest use of the drawn format as a pedagogical aid. They argue 
that other formats such as oral presentation or motion portrayed by 
photography or computer graphics also might reduce memory overload 
and assist with representing and organizing problem data. 
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Abstractor's nanents 

The study was a "clean" study in that it used an adequate sample, 
dealt with clearly-defined constructs such as problem formats, and 
extended the results of other studies In this chain of inquiry. The 
article was clearly written, making it easy for the reader to grasp 
exactly what was done in the study. 

I raise two questions concerning procedures of the study. One 
concerns the use of only two categories, either "high" or "low," 
for reading ability. Would the results be the same if the upper and 
lower quart iles were used to define high and low reading ability? 

A second question concerns the lack of information about subjects' 
prior experience with the three problem formats. To what extent did 
instruction fccus on one or more of the three formats? Presumably 
this would influence performance on problems in those formats. 
Although I suspect that typical textbooks were used and the drawn 
format was encountered least frequently, a description of relative 
emphases on the three formats would further clarify the findings. 

Only one suggestion for further research is offered in the 
article, that being inferred from the authors* discussion that other 
formats may offer advantages similar to those of the drawn format. 
I think another extension of this research would be a study designed 
to gather information on why the drawn format was superior. 
Carefully designed clinical interviews might be used to determine 
specific aspects of the drawn format that contribute to Improved 
performance. This would help to clarify the extent to which working 
memory overload is related to problem difficulty in the various 
formats. 
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Another issue which this study raises is the appropriateness 
of these formats for instruction at various stages in students' work 
with story problems. Is the drawn format superior to the others when 
initially presenting new and difficult story problems? After students 
have had substantial experience with story problems of a certain 
type? The evidence provided by the simpler tests at Grades 4 and 5 
seems to suggest that the format effect may be similar for problems 
at different levels of difficulty. 

• « 

This study is a likely one U cite in answer to critics who 
contend that mathematics education researchers have nothing to offer 
to practitioners. The findings of this study shed light on an issue 
directly related to pedagogy and the design of instructional materials. 1 
Despite its frequent use in textbooks, the telegraphic format appears 
not to enhance student performance, even for students classified as 
below average in reading ability. Furthermore, this study suggests to 
teachers and .materials developers that the drawn format can enhance 
student performance, making a difficult topic somewhat easier for 
students. 
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Pallas, Aaron Jjl. and Alexander » Karl L« SEX DIFFERENCES IN 
QUANTITATIVE. SAT- PERFORMANCE: NEW EVIDENCE ON THE DIFFERENTIAL 
COUTvSE HYPOTHESIS. American Educational Research Journal 20: 165-182; 
Summer 1983. 



Abstract and comments prepared for I.M.E. by C. JAMES LOVETT, 
Brooklyn College of the City University of New York. 

r 

1. Purpose 

This study was motivated for the most part by the Benbow and 
Stanley report in Science (210: 1262-1264; December 1980). The 
purpose was to "provide a direct test of the hypothesis that the sex 
difference in quantitative SAT performance may be due to differences 
in the pattern of quantitative coursework taken by males and females 
in high school" (p. 165). 

2. Rationale 

The report notes the sex differences on tests of mathematical 
aptitude and achievement documented by various studies, and the 
attention given to social-psychological factors to account for these 
differences. Primary emphasis is placed on the Benbow and Stanley 
report which challenged the differential coursework hypothesis as an 
explanation for sex differences in mathematical ability. The report 
is criticized mainly because (1) it was not a direct test of the 
hypothesis in that it considered students prior to the opportunity for 
differential coursework and (2) because of the dubious generalizability 
of findings based on "a self-selected group of extremely able, highly 
motivated students" (p. 169). The need for the study is based on these 
weaknesses of the Benbow and Stanley study and the further observation 
that "no one has come forth with convincing evidence supporting the 
differential coursework hypothesis" (P» 167). 
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3. Research Design and Procedures 

The data were derived from an earlier study (conducted by the 
Educational Testing Service), and consisted of: student sex, parent's 
occupations, race, 9th grade SCAT-Q score (obtained in 1965), 
enrollment in high school mathematics courses (13 categories, obtained 
from academic transcripts), enrollment in three non-mathematics 
quantitative courses (physics, business/commercial, industrial arts), 
grade point average for the above courses, and 12th grade SAT-M score 
(obtained in 1968), The sample consisted of 1842 females and 1770 
males. The statistical method was a multiple regression analysis of 
the SAT-M scores on the background and quantitative coursework 
variables. 

Citing a personal communication, the authors note a disagreement 
between themselves and Benbow and Stanley regarding the appropriateness 
of the SCAT-Q as a "control for pre-high school differences in 
standardized test performance when SAT performance is used as the 
criterion" (p. 172). Benbow and Stanley apparently take the position 
that the characteristic measured by the SAT-M ("mathematical reasoning 
ability") is not measured adequately by the SCAT-Q. Citing CEEB and 
ETS studies, the authors provide a detailed argument to support their 
position that the SCAT-Q is indeed a suitable control variable. 

4. Findings 

A. Findings from descriptive statistics 

1. With respect to parent's education, race, and ninth-grade 
SCAT-Q scores, no significant sex differences were found. It 
is noted in particular that pre-high school quantitative 
• performance (measured by the SCAT-Q) of males and females is 
about the same (males 290.1; females 289.7) in contrast to 
the Benbow and Stanley result (using the SAT-M). 
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2. "... Females are much leas likely than males to take 
certain higher level mathematics and quantitative science 
courses." (p. 175) 

3. The grade point average for the courses they did take 
were "somewhat" higher for females than for males. 

4. There was a significant sex difference for the twelfth- 
grade SAT-M (males 425.2; females 388.5). 

B. Findings from the regression analyses. 

Three equations were considered: (1) a base-line equation with 
mother's and father's education, race, sex, and SCAT-Q as predictors; 
(2) an equation with the five previous predictors together with the 
16 coursework variables; (3) the previous 21 variables together with 
grade point average. 

The first equation indicated an adjusted average difference on the 
SAT-M scores of 35 points in favor of the males. Given the similarities 
noted between males and females on the five predictor variables, the 
authors expressed "...considerable confidence that the differences... 
in the 12th grade SAT-M performance did in fact emerge during high 
school" (p. 177). The resulting R 2 was .569. The second equation 
reduced the gap in SAT-M performance to 14 points, and > .! ^ R to 
.692. "This indicates that most of the initial 35-point differential 
may be due to ser.-linked differences in student's high school programs" 
(p. 178). The result indicated the plausibility of effect due to the 
pattern of coursework with larger benefits noted for enrollment in more, 
advanced courses. The third equation indicated a substantial effect 
for grades suggesting that "...level of mastery of the material 
encountered in quantitative coursework has an important influence on 
SAT-M performance over' and above that simply coming from having taken 
such courses" (p. 179). This equation indicated an adjusted average 
difference in SAT-M scores of 20.5 points and R 2 of .707. 
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5. Interpretations 

Considerable support Is given for the hypothesis that sex 
differences in high school programs of study are "highly relevant" 
to the SAT-M differences which emerge by the end of high school. Whilt 
differential coursework and grade performance do not fully explain 
the gap in SAT-M scores, it is premature to conclude that the residual 
difference is not due to socialization processes. 

Abstractor's Comments 

This study is reasonably well designed, conducted, and reported. 
I am bothered by the use of data collected 15 to 20 years ago, but 
my concern is somewhat balanced by the apparent soundness of the data 
and the fact that large-sample, longitudinal data is difficult and 
expensive to obtain. Based on the methodology and findings presented, 
1 find myself in general agreement with the authors 1 conclusions. 

The report is a worthwhile contribution to the research literature 
on gender /mathematics- learning issues. I think, however, that for the 
overall improvement of mathematics teaching as well as for better 
understanding of the underlying learning processes, further research 
along this particular line (testing of the differential coursework 
hypothesis) is not necessary. The real rebuttal to Benbow and Stanley 
(1980) lies not in further research, but in pointing out the 
inadequacies of their methodology and, much more important, the 
interpretations they give to their findings. This, I think, has been 
accomplished (See Letters, Science, 212: 114-121, April 1981); now 
let us turn to more critical matters. 
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Powell. 3. and Steeltnan, L. C. VARIATIONS IN STATE SAT PERFORMANCE: 
MEANINGFUL OR MISLEADING? Harvard Educational Review 5U: Jo¥-*U/, 
November 1984. 

Abstract and comments prepared for I.M.E. by ALAN OSBORNE and 
MARGARET SOOY , Ohio State University. 



1 . j'urpose 

The primary objective of this research is the examination of stat 
and regional variations in SAT performance. Factors such as percent 
of seniors taking the test; demographic/compositional factors such as 
sex, racial composition, and econom<: status; and school structural 
characteristics such as number of years of coursework in social 
sc iences, natural sciences, and humanities, public expenditures per 
student, and public vs. private school are examined for their 
contribution to variation in SAT performance. Will taking them into 
account alter state or regional comparisons of performance? 
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Rat ionale 



Not since the launching of Sputnik in 1957 has so much attention 
been focused on the American educational system. Particular attention 
has been paid to the decrease in scores on the Scholastic Apptitude 
Tost (SAT) that measure preparedness for college and are used as 
admission criteria by some colleges. In the past twenty years the 
average total SAT scores have decreased from 97 3 to 893 out of a 
possible 1,600 points. 

Standardized test scores are used as "objective" indicators of 
academic growth and potential, to evaluate differences among 
ino viduals, and as an aggregate measure of the national academic 
health. State and federal officials compare state scores and speculate 

about causes and correlates. 
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Since educational institutions are governed chiefly at state and 
local levels, an analysis of test performance at the state and 
regional levels is needed. With the publication of state "report cards 
in educational performance by the media and state and federal 
officials, implications of state variation in test performance extend 
beyond the realm of academic assessment and into the economic sphere. 

The SAT was analyzed because (a) it is taken by about two-thirds 
of the entering students, and (b) it is the test most frequently used 
by scholars, politicians, and the lay public to assess school quality. 

3. Research Design and Procedures 

Statistical reports, including information on the average SAT 
scores, verbal and mathematical sections of the SAT, and statistical 
descriptions of the test-takers for each o* the fifty states, were 
requested and received from the Educational Testing Service. These 
reports included student self-reports on academic and financial 
background. 

Tht. state was the unit of analysis. There were fifty equally 
weighted observations in this study. The dependent variable was the 
total SAT average, but mathematical and verbal scores were evaluated 
separately. The operational definitions of the independent variables 
were: 

percentage - percentage of eligible high school seniors in a 
state taking the SAT. 

sex composition - percentage of test-takers in a state who were 
^ female. 

racial composition - percentage of test-takers in a state who 

were black, 

median income - the median family income of the test-takers in a 

state. 
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percent public - percentage of students who reported attendance 

In public as opposed to "other than public" 
schools at the 'time of the examination in a state. 

academic years - the mean number of years spent studyirg social 

sciences, natural sciences, and humanities. 

expenditures - public expenditures per student in a state. 
The data were analyzed in sequential Stages : 

Stage 1. The impact of the percentage o;f test-takers on SAT scores 

Stage 2. The impact of features of th(^ test-talcing population 
"such as sex composition, racial composition, and median income. 

Stage 3. The impact of private versus public education, time 
spent in basic subjects, and public expenditures. 

The data were reanalyzed to examine verbal and mathematical scores 
separately. The affects of these variables were estimated using 
ordinary least-squares regression equations. 

4, Findings 

•Nearly three-fourths of the state variation in SAT scores can be 
attributed to the percentage of students in each state who take the 
test. Examination of a scattergram that relates the percent of 
graduating seniors in each state taking the SAT and the average 
total SAT scores of each state showed a non-linear relation. Adiing 
the square root of the percent to the model- increased the explained 
variation to 81 percent. 

Unadjusted SAT averages underestimated the South/non-South 
variation. The disparity between the averages of southern and 
non-southern states increased from thirty-four points to forty-six 
points with the corrections. 
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In the second stage of analysis, adding the variables sex 
composition, racial composition, and family income bolsfcsted the 
explained variance to .922. The effect of racial composition was^the 
most pronounced. 

The relative position of southern states improved if these factors 
were controlled, but the regional discrepancy remain^ significant 
beyond the .01 level. 

Using school/structural variables, the explained variance 
increased to .944. All demographic variables remained significant 
with the exception of income. Of the structural characteristics, only 
expenditures attained significance. Results did not indicate that 
there is a beneficial impact of private schooling on test performance. 
The regional disparity was reduced to a nonsignificant level. 

Analyses of f-he verbal and mathematical scores demonstrated results 
similar to the total SAT analyses. The only structural variable with 
significant effect is expenditures. 

For mathematics scores, the South/non-South difference increased 
with percentage controlled. Sex was a significant predictor if all 
demographic variables were included. Regional difference remained 
significant. The only school/structural characteristic that remained 
significant was expenditures. The South/non-South differ«nce 
persisted if all of the variable? -*-A*e controlled. 

5. Interpretations 

The larger part of the variance in state differences in SAT scores 
is attributable to the percentage as well as the compositional 
features of test-takers and does not reflect differences in the 
organization of statr school systems. Comparisons of states not 
accounting for these factors may be seriously misleading. 
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Most state variation is a function of one factor, the percentage 
of eligible students taking the exam. The strength of this factor may 
be a consequence of a combination of selection biases related to 
percentage including ability, socioeconomic background, and 
educational aspirations. Sex composition affects a state's level of 
SAT performance only through its influence on mathematical proficiency, 
while racial composition affects both verbal and quantitative 
performance. States that encourage women and minorities to attend 
college may unwittingly dilute their uncorrected scores. 

The number of years spent in academic coursework is not a 
significant predictor of state differences. These findings do not 
support the National Commission on Excellence's (1983) recommendation. 
There may be several reasons for this. First, students devoting more 
time to basic coursework may be those with motivation and ability who 
take the exam. Second, the average number of years in basic subjects 
may not vary enough to result in state differences. Finally, since 
the SAT evaluates competency on skills that are usually taught in 
early high school years, the addition of academic years beyond this 
point may be inconsequential in terms of state SAT differences. 

The only school/structural characteristic tested in this model 
that seems to influence state/regional disparities in SAT performance 
is public expenditures. The greater the per student funding, the 
higher the state's level of SAT performance. Both verbal and 
mathematical performance spears to be associated positively with 
this factor. This contradicts recent claims that expenditures have no 
effect and attests to the need for using corrected SAT scores. As 
long as states vary in their capacity or willing ss to allocate funds 
to education the disparity in state SAT scores may continue or be 
made worse. 
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The regional gap on total SAT performance lessens if demographic 
and structural characteristics are taken into account. These findings 
imply that if the credentials or skills required to enter occupations 
continue to shift to a more technical base, then students educated in 
the South may compare poorly to others in the labor force. If the 
population of the United States continues to drift southward, a 
further decline in SAT performance may be expected. 

Educational performance is a multidimensional concept. 
Overreliance on any one indicator, such as the SAT test, to draw 
sweeping conclusions is unwise. Future researchers should broaden 
this research with the inclusion of other variabler and with the 
assessment of other measures of educational effectiveness. 

Abstractor's Comments 

Evaluation is ultimately a political a^t. Each year the popular 
press considers the results of the SATs, making comparisons and 
comments about the nature of schools and schooling. Moreover, 
reactions to the SAT data are not limited to lay people and the 
popular press; frequently our colleagues in education are puzzled 
about the meaning of SAT scores, the appropriate interpretations 
to place on them, and what the implications are for schools and 
schooling. Mathematics educators are frequently called on to comment 
on the nature of testing results. 

Powell- and Steelman have provided a major service in making 
explicit some of the statistical artifacts inherent in SAT testing 
ana* comparisons. Their careful analysis and commentary should make 
mathematics educators more sensitive to sampling problems in research 
or evaluation projects. Some of the vfhsj,a#{es we might like to ignore 
do make a difference when we attempt to draw conclusions. 
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The fact that public expenditures was the only school/structural 
characteristic that seemed to Influence disparities in SAT performance 
is an important difference. Examination of unadjusted scores does 
not reveal an effect of such expenditures. The consequences of 
erroneous conclusions that can result from consideration of unadjusted 
scores are evident. 

This was a well-done analysis of existing data. Such secondary 
analysis is useful to mathematics educators in dealing with the nature 
of evaluation in its political aspects. We think that the "score 
card of the states" that is used to assess state-by-state performance 
across the country by the Department of Education (see the February, 
1985 issue of Phi Delta Kappan) indicates clearly the political stake 
for mathematics educators. 
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AN EXAMINATION OF THE EFFECTIVENESS OF A CROSS-AGE TUTORING PROGRAM IN 
MATHEMATICS FOR ELEMENTARY SCHOOL CHILDREN. American Educational 
Research Journal 20: 103-111; Spring 1983. 

Abstract and comments prepared for I.M.E. by BARBARA SIGNER, University 
of Massachusetts-Boston. 



1. Purpose 

The purpose of this study was to conduct empirical research on the 
cognitive and affective effects of cross-age tutoring in mathematics 
with elementary school children. Both the tutors and tutees were 
studied for possible consequences as* a result of such a program. 
While other mathematics learning was examined, the study focused on 
effects on learning "operations" of mathematics. In addition, the 
study sought to investigate whether there are significant differences 
among the tutees based on the predetermined mathematics achievement 
level (high or low) of the tutors. 

2. Rationale 

The rationale for this study is that most of the peer-teaching 
evaluation studies reported in the literature are based on anecdotal 
reports rather than the analysis of Sbjectlve data. With the positive 
claims made in these previous studies, the authors sought to conduct 
a rigorous experimental study to investigate these findings. 

In regard to the few experimental studies conducted on the 
efficacy of academic and social benefits of cross-age tutoring, the 
authors' review of the literature found many committed violations of 
empirical research design. Therefore, the authors were doubtful of 
the authenticity of the positive claims reported and designed a 
rigorous experimental study to substantiate claims or report the 
absence of experimental support. 
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3. Research Design and Procedures 

Four experimental groups were formed for the study. They were 
based on the premeasured mathematics achievement levels of both the 
tutors (fifth and sixth graders) and tutees (second and third graders): 

(1) high achieving tutor/high achieving tutee 

(2) high achieving tutor/ low achieving tutee 

(3) low achieving tutor/high achieving tutee 

(4) low achieving tutor/low achieving tutee. 

^Subjects that participated in the experimental groups came from 
tjjo different schools. Four classes from one school (experimental 
school) and two classes from a second school (experimental/control) 
provided both male and, female tutor/ tutees . 

Subjects for a control group also were chosen from two separate 
schools * Two classes in the experimental/ control school also provided 
a nontutoring class of sixth graders and a nontutored class of third 
graders to control for within-school effects, A third independent 
school (noncontact control school) provided four nontutoring classes. 
They consisted of classes in the following grades: sixth and fifth, 
third and second. 

4. Findings 

Results of the study were the following: 

(1) Mathematics achievement of the tutors increased significantly 
more than that of nontutors. 

(2) Mathematics achievement of the tutees increased significantly 
more than that of the nontutees. 

(3) The predetermined mathematics achievement level of the tutors 
did not affect the mathematics achievement gains of the 

tut ees. 
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(4) There were no significant increases in self-esteem scores for 
tutors over nontutors. 

(5) Tutors increased the "operations" and "other** (estimation, 
geometry, fractions and time) scores significantly more than 
nontutors. 

(6) Tutees showed significant gains in "operations" but not in 
"other" scores over nontutees. 

5. Interpretations 

The primary interpretation of the results of this study is that 
empirical research does support the previous anecdotal claims of the 
mathematics achievement benefits of cross-age tutoring for tutors and 
, tutees in the tutored areas. In addition, gains in achievement for 
tutors in the nontutored areas can be documented. The researchers 
interpret this to be the result of an experience that increased 
motivation to study mathematics. 

Abstractor's Comments 

The strength of this study lies in its rigorous planning and 
execution. The research design carefully eliminates intervening 
variables and conservatively documents all claims. The benefits of 
cross-age tutoring to both tutors and tutees as shown in this study ai 
important for mathematics educators to be aware of and to research 
further. 

While both males and females participated in the study, as both 
tutors and tutees, there is no indication that pairings by sex was 
investigated. It would be of interest to know the , following: the 
number of pairings of tutor and tutees by sex (same sex, male tutor - 
female tutee, and female tutor - male tutee). In addition to the 
numerics] breakdown of the pairings by sex, the gains in mathematics 
achievement according to the specified pairings would be Important to 
know. This could be a possible area to investigate in the future. 
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Abstract and comments prepared for I.M.E. by JOHN C. MOYER, Marquette 
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1. Purpose 

4 

The purpose~~oFtTtis research is stated only in a very general way; 
"...to look at problem solving as it relates to certain types of word 
problems found in elementary school mathematics." The design of the 
study, however, belles a more specific purpose, viz to investigate 
causes for the performance of those students who are very good at 
solving one-step word problems but are very poor at solving similar 
multi-step problems. 

2. Rationale 

The impetus for this study seems to have been provided by the 1977 
and 1981 British Columbia Mathematics assessments. In particular, the 
author deduced from the results of the B.C. assessments that there is 
a large group of students who are able to solve one-step, textbook 
word problems, but are unable to solve related multi-step problems. 
He then designed this experiment in which he identifies such students 
and compares their problem-solving performances to those of students 
who are able to solve multi-step problems. 

3. Research Design and Procedures 

The study was conducted in two parts. The first part was an 
identification phase in which 18 elementary school students were | 
selected for individual interviews. The second part was an interview 
phase* 
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For the first phase all grade seven students enrolled in seven 
different British Columbia elementary schools (N - 283) were given a 
42- item computation test and a 12- it em word problem test. For the 
word problem test, 600 seventh-grade items were collected by the 
author. From these items, 120 one-step and related multi-step 
problems appropriate to grade seven were derived and field tested. 
Three forms of the test were created. Each form was composed of four 
sets of three problems. Each set consisted of: (1) a one-step 
problem, <2) a related multi-step problem whose statement explicitly 
contained all the data needed for the computation, and (3) a related 
multi-step problem in which one of the numbers needed in the 
computation had to be generated from the given numbers. The 
computation test was designed to assess the ability of the students 
to perform the computation needed to solve the word problems. 

The following criteria were used to select students for the 
interview phase of the study: each student selected was required to 
correctly complete all four one-step problems, to have a score of 75% 
or better on the computation test, and to have a reading level of at 
least 7.3. These criteria eliminated all but 54 of the 283 students. 
- The further criterion that the students score either extremely poorly 
(0-3) or extremely well (7-8) on the multi-step problems eliminated 
another 25 students. From the remaining 29 students, those with the 
9 lowest scores on the multi-step problems were placed in group A and 
those with the 9 highest scores were placed in group B. Both groups 
averaged 83% on the computation test, had a reading level of 8.7, and 
scored a perfect 4 on the one-step problems. Group A, however, 
averaged 3.0 on the multi-step problems, while group B averaged 7.1. 

During the interview phase each of the 18 chosen students was 
asked to think aloud as he or she worked the problems. Three sets of 
three related problems were used. However, each student was given 
only two of the three sets to solve. The presentation of problem sets 
was systematically varied so that each problem set was given to six 
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students in group A and six students in group B. The students received 
a one-step problem followed by two of the first type of multi-step 
problems, then the other one-step problem followed by the other two 
multi-step problems. Data collected for each problem can be deduced 
from the results section which follows. 

4. Results 

(a) Group B outperformed group A on both parts of the test (100% 
vs. 88.9% on the one-step problems and 86.1% vs. 58.3% on the 
mt^fcfc-step problems) . 

(b) Average time spent (ignoring outliers) on all but two of the 
problems was greater for group A than for group B students. 

(c) Group A students re-read problems more frequently than group 
B students (Group A: 47 rereads; Group B: 20 rereads). 

(d) Group A students re-attempted 7 of their 17 missed problems. 

(e) Only three missed problems involved computation errors. 

(f) Group A students checked only two of their answers and group 
B students checked only nine of their answers; and 11 checks 
resulted in correct solutions in every case. 

(g) Of eight^coramonly observed heuristics (diagrams, trial and 
error, related facts, related method, mnemonics and equations, 
algorithms, special cases, and successive approximations), 
only "uses algorithms" was observed to any great extent, but 
its use was "unanimous." 

5. Interpretations 

In the discussion of these results the author comes to four main 
conclusions: 

(a) The all-pervasive strategy of searching for key words, 

selecting an algorithm (or sequence thereof) and applying the 
algorithm(s) was full of pitfalls: the incorrect algorithm(s) 
were chosen (7 instances), incorrect inputs were used (9 
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instances), and only part of a multi-part sequence of 
algorithms was identified and'/ or completed (8 instances). It 
is to this last pitfall that the author attributes the 
observed discrepancy between performance on one-step and 
multi-step problems. He asserts that 40% of the incorrect 
answers to the multi-step problems came about because the 
students treated the problems as one-step problems. He feels 
that such errors could be avoided by encouraging students to 
re-read the entire problem once key words are found. 

(b) Students need to become convinced of the importance of 
checking their solutions. Of the 22 problems worked 
incorrectly, none was checked. 

(c) Students do not make much use of heuristics other than "uses 
algorithm." However, for these type problems it is possible 
that nothing else is necessary. 

(d) The students were willing to work hard and long on the 
problems, perhaps because the prob 1 :ms were well within their 
ability level, the arithmetic was easy, and/or the students 
liked the interviewer. 

Abstractor's Comments 

Much talk recently has centered around the need to spend less rime 
in the elementary school on computation and more time on problem 
solving (as well as other topics). Unfortunately, problem solving is 
not currently held in unanimously high regard by teachers or students. 
Part of the reason lies in the lack of success that is evidenced when 
problem solving is taught. So, it seems reasonable to conjecture that 
studies that help unlock the mysteries surrounding word problems will 
go a long way toward getting problem solving into the classroom. 
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s The fact that multi-step problems are more difficult for elementary 
school students to solve than similar one-step problems is a 
well-accepted^ fact. However, as Sherrill rightly points out, it is 
quite disconcerting to discover that students who are very good at 
solving one-step problems can be so very bad at solving their 
multi-step counterparts. The existence of such students points to 
some type of discontinuity in the way these problems are being 
processed. The whole is more difficult than the sum of the individual 
parts. 

Sherrill has designed a very nice study to investigate this 
phenomenon. With the exception of a somewhat sketchy description of 
the interviews (did he have hints, ask questions, etc?), he has clearly 
and succinctly reported the method he used. He even included a 
listing oft :he problems he used during the interview phase of the study. 

In carrying out the study, he has admirably achieved his goal of 
selecting two groups of students who are matched almost perfectly on 
the important traits of reading ability, computation ability, and 
ability to solve one-step problems, yet are at opposite ends of the 
spectrum on ability to Bolye multi-step problems. 

He has reported appropriately the data regarding student success 
( right /wrong ) , time to completion, number of re-reads, and number of 
re-attempts. The raw data are presented in tabular format broken 
down by problem and group. For student success, he further breaks 
down the groups, tabulating right or wrong for each individual on 
each problem. 

The data on the arithmetic errors, frequency of answer checking, 
and use of heuristics is less complete. He no longer presents the 
data in tabular format. Further, he does not break down the data by 
problem. Further still, except in the case of the arithmetic errors, 
he does not give separate results for Groups A and B. The reader 
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begins to wonder why the author has chosen to abandon the major 
strength of the design— the ability to compare two matched groups of 
students on 'the important variable where they differ— ability to solve 
multi-step problems. 

In the discussion section, the reader wonders even more about the 
reason for abandoning the two-group comparison. At no point in the 
discussion or in the subsequent conclusions does the author even 
mention the existence of the two groups, let alone come to any 
conclusions regarding reasons for the difference between their abilities 
to solve multi-step problems. Furthermore, the discussion centers 
around that data which were less completely presented. 

For example, Sherrill's main conclusion is that multi-step 
problems are missed because they are treated as one-step problems. 
The evidence he uses to support this conclusion is that 40% (8 out 
of 20) of the incorrect solutions to the multi-step problems were 
incorrect because the students treated the problems as one-step 
problems. Now we know from the data that Group A students missed 
15 multi-step problems and Group B students missed 5. We are not 
given a similar breakdown by group with respect to tb" 8 problems 
that support the main conclusion. Even if one were to agree that 40% 
is a large enough percentage to support Sherrill's conclusion, one is 
still left wandering how much of the 40% was attributable to group A 
students. Perhaps even more puzzling is the reason for Sherrill's 
decision to omit the information. 

Sherrill suggests that if students were encouraged to re-read 
problems, then they would notice that the problems were multi-step and 
treat them as such. He does not cite anything from the study to 
support this suggestion. Rather, he seems to be drawing on 
conventional wisdom. Unfortunately, the data from the study do not 
support this suggestion, since group A students re-read every 
multi-step problem more often than Group B students, yet Group A %as 
outperformed by Group B. 
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Sherrill also suggests that students would correctly solve 

multi-step problems if they checked their work. He states: 

Of the 108 problems used in the interviews, 11 were checked. Of 
the 11 problems checked, all were solved; so, of course, of the 
22 problems worked incorrectly, none were checked. The data 
suggest strongly that students must be convinced of the 
importance of checking their work and the final solution. 

Are we to infer that Sherrill is basing this recommendation on the 

fact that better students check their answers more often than poorer 

students? If so, why doesn't he mention this and remind us of the 

data to support the recommendation? In this instance Sherrill has 

previously given us data which tell us that Group A students checked 

2 problems and Group B students checked 9 problems. Unfortunately we 

are not told how many of the problems checked were multi-step problems. 

It would also be helpful to know how many times the checking revealed 

errors that were subsequently corrected. 

These are very curious ommissions, for without such information 
it is difficult to judge the force of Sherrill' s recommendation. For 
example, we might ask how Sherrill explains Group B's ability to solve 
multi-step problems without checking. This question is critical only 
if Group B was, in fact, able to solve lots of multi-step problems 
without checking. But we do not know whether Group B students 
checked any of the multi-step problems at all. If they did not, then 
they correctly worked 31 problems without checking. If we assume that 
all the checked problems were multi-step, then they only worked 22 
problems without checking. Even if we assume that all the checked 
problems were multi-step, we don't know whether the checking really had 
any effect on the eventual outcome. So the reader is left wondering 
whether the Group B students were able to solve 22/36, 31/36, or ?/36 
multi-step problems without checking them. 

In the beginning of the article, Sherrill makes a distinction 
between the two different types of multi-step problems that are used 

in the study: tfJ? first type had all the data needed for the 
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computation explicitly given; the second type required the solver to 
generate some data before it could be used in the computation of the 
final answer. The distinction seemed well worth considering. The 
distinction, however, seems all the more worthwhile in the light of 
Sherrili' s major conclusion that the main difficulty with multi-step 
problems is that whey are treated as if they are one-step. If such is 
indeed the case, one might expect that there would be differential 
performance -on the* two types of multi-step problems, with the second 
type being more likely to be treated as if '.hey were one-step. 
.Unfortunately, Sherrili not only fails to discuss the implications of 
the differential performance; he does not even acknowledge the 
existence of two types of problems once he begins the Results section. 

It is very unfortunate that Sherrili does not report more of 
the qualitative differences between Group A and Group B students, 
rather than these somewhat incomplete quantitative differences. In 
■ k • idles of this -type, with such a small number of students, much 
more can be grined by reporting on the qualitative ways that good 



approach used by the poorer students. This study is too well 
conceived, designed, and carried out to allow that information to go 
unreported. Even if the result was that there is no identifiable 
difference between the groups, we would know that we should look 
elsewhere. Perhaps improved performance on multi-step problems is 
simply £* matter of time spent and experience gained. 



students approach problems and compare them to the qualitative 
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Slavin; Robert E. ; Leavey, Marshall B. ; and Madden, Nancy A. 
COMBINING COOPERATIVE LEARNING AND INDIVIDUALIZED INSTRUCTION : 
EFFECTS ON STUDENT MATHEMATICS ACHIEVEMENT, ATTITUDES, AND BEHAVIORS. 
Elementary School Journal 84: 409-421; March 1984. 

Abstract and comments prepared for I.M.E. by ROBERTA L. DEES, Purdue 
1 University Calumet, Hammond, Indiana. 

1, Purpose 

The authors ask, "How can it be that individualized, programmed 
instruction ^does not increase student achievement?" (p. 410) and 
"Can the problems of programmed instruction be solved in a way that 
would make this strategy effective?" (p. 411). 

The two studies described compare a program in which 
student -managed individualized instruction is provided, with 
"traditional methods" for teaching mathematics, _ 

2. Rationale 

The problems that individualization, or programmed instruction, 
attempted tc address are still with us and are getting worse as 
students are being mainstreamed and/or desegregated. Programmed 
instruction, while logically compelling, has never been shown to be 
.effective, apparently because of difficulties in management and 
motivation. Since cooperative learning methods have been found to be 
effective, a method called Team-Assisted Individualization (TAI) 
was designed. This program is managed by the students themselves, 
and motivation is fostered by team recognition based on team members' 
progress through the program. 

The curriculum materials themselves were constructed to be 
checked by teammates with answer keys provided; skills were 
subdivided so that a ehe:k was required every four problems. Students 
presented themselves for a test on a skill only if they passed the 
"checkout" for that skill. 
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3, Research Design and Procedures 

This article reports two studies in elementary schools. In 
Study 1, the TAI system is compared with a second treatment and with 
a control. The second treatment consisted of Individualized 
Instruction (II), exactly the same curriculum materials and manage- 
ment program, but without the teams. That is, the students also 
received answer keys with their units but did their own checking 
individually. Tests were still given by rotating student monitors. 




In Study 2, the' TAI system is compared with a control. In both 
studies, the control group "used traditional methods for teaching 
mathematics, which consisted in every case of traditional texts, 
whole-class lectures, and group-paced instruction, supplemented by 
small, homogenous, teacher-directed math groups." (p. 413) 

Study \ i The three treatments were implemented for eight weeks 
in the spring of 1981 in a third-, fourth-, and fifth-grade class in 
each of six schools in a middle-class suburban Maryland school 
district. Of the 504 students, 23% were receiving special education, 
reading Instruction or other special educational services. 

Study 2; Subjects were 375 students in grades 4, 5, and 6 in 
another suburban Maryland school district; 27% were receiving special 
educational services. Two TAI schools with a total of 10 classes 
were matched with six control classes in two schools. In this study, 
TAI was implemented for 10 weeks in Spring 1981, and compared to the 
control. 

Students in both studies were measured in pre- aod posttests on 
mathematics achievement, ' attitudes, and behavior ratings. For 
mathematicVachievement, the computation subscale of the Comprehensive 
Test of Basic Skills (CTBS), level 2 (1975) was used. For attitudes, 
two eight-item scales were given, one on "liking of math class" and 
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the other on "self-concept in math." Teachers rated subsamples of 
their students on behavior, using these subscales: classroom 
behavior, confidence, friendships, and negative peer behavior. 

6. Fin dings 

Outcomes on seven criteria were reported: mathematics achievement 
as measured by gain on the C&S, gains in self-scores on "liking of 
math class" and in "self-concept in math," and decreases in the 
number of problems reported in the four teacher-rated behavior 

subscales, ( 

« 

Students in the individualized instruction programs significant ly t 
outperformed the control groups on the CTBS and on the self-confidence 
and friendship subscales of the behavioral rating. Other results 
were mixed. * 

In Study 1, a marginally significant overall treatment effect was 
found on the CTBS (p C .07). The TAI group gained significantly more 
than the control group (p < .03), while the II group gained marginally 
more (p < .09). When compared pairwise, there were no significant 
differences between the TAI and II groups. In Study 2, the TAI group 
performed significantly better than the control group on the CTBS 
(p< .03). 

In the six affective measures, students in the treatment groups 
in Study 1 were significantly higher than the control groups. 
Between the two treatments (TAI or II alone), there were no 
significant differences except in one scale, the self-confidence 
subscale of the behavior rating (p<.03). In Study 2, a similar 
analysis showed significant differences between the treatment group 
and control group on only two of the affective scales, both subscales 
of the teacher-rated behavioral scale: self-confidence (p<.02) and 
friendships (p<.01). 
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5. Interpretat ions 

The authors feel that the evidence supporting the positive effects 
of the TAI program on mathematics achievement is "relatively 
unambiguous" (p. 418). Nevertheless they cannot conclude that the 
use of teams adds to the achievement effects of the II program alone. 

The improvement in students 1 attitudes toward mathematics in the 
first study was not replicated in the second study. 

The authors suggest caution in interpreting the results of the 
behavioral ratings of the teachers, fcho may have had trouble main- 
taining objectivity during the experiment. 

They note that the curriculum materials were finely subdivided 
(20 items for each step) and provisions were made for feedback early 
and often (students worked on a block of four items and proceeded 
to the next block only if those four items were correct.) 

Perhaps the most important finding of this study was that an 
individualized mathematics program was developed that could be 
managed by a single teacher without an aide, relying on the 
students themselves to perform the routine checking and 
management procedures of the program, (p. 421) 

They further conclude that students themselves may be enlisted to 
help solve the management and motivational problems inherent in an 
individualized method. 

Abstractor's Comments 

The authors have suggested the limitations of the study 
(restricted generallzability, due to using the individual as the unit 
of analysis; possible Hawthorne effect; and the short duration of the 
experiment). I make three substantive comments. 
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1. The title is misleading, because the authors use "cooperative 
learning" vhen they mean "cooperative management." It is not a 
typographical error, but a misunderstanding of the term. In 
describing their method in some detail, they explain that though the 
students are working in pairs, "students read their instruction sheets 
individually, asking teammates or the teacher for help if necessary." 
(p. 412). Thus the purpose of working In pairs is to exchange answer 
Sheets and check each other's work. In fact, the students are 
generally not working together nor even necessarily working on the 
same thing. This is not considered to be "using cooperation as a 
learning method." In such a method, studevte would be working 
together on the same thing, and would produce a common end product 
(for example, they would agree on the answers before checking with 
the answer key) (see Johnson and Johnson, 1975). 

The authors' misunderstanding of the method is further exemplified 
in this statement: "... cooperative learning methods are group 
paced; the entire class studies the same material at the suae rate." 
(p. 411) While that could happen, it is by no means a necessary 
characteristic of a cooperative learning method; an advantage of the 
method is that different small groups can be working on different 
things. 

Had the authors used a true cooperative learning method, they 
might have seen even more phenomenal gains, as I have when I have 
insisted that students work together and discuss mathematics. Talking 
about mathematics can enable students to clarify concepts and increase 
their problem-solving ability (as opposed to computational skills) 
(Dees, 1983). 

2. It is a shame that these researchers feel the need to reward 
students for doing what they should be doing and should want to be 
doing, namely learning. The chance to work in a social atmosphere 
with peers was probably motivating enough. And research by 
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psychologists has shown that intrinsic motivation, when it exists, 
may actually be* lessened or destroyed if external rewards are added. 
Therefore I question the need for and the use of the certificates 
for teams with the highest scores. 

3. I feelj that programmed instruction was abandoned too soon. I 
was involved in a successful individualized program for several years. 
Students often gained two grade levels in one year. But, as the 
authors observe, students find its continual use tedious, boring, 
and lacking in human contact. Students sometimes manufactured 
questions to ask, just to, get a chance to talk to another human being. 
The authors are right when they try to get student interaction back 
into the learning process in mathematics. 

Our problem is looking for the one method that works best. We 
need to use a (combination of methods; one method works better for one 
goal and for some students, another for others. And no student wants 
to do the same thing day in and day out for the entire school year. 
The authors havp demonstrated that it is possible to design an 
Individualized Instruction program in which students can help to 
manage their own learning. Such a program should be included in 
teachers' lists of effective teaching stragegies. 
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Abstract and comments prepared for I.M.E. by ROBERT C. CLARK, Florida 
State University, Tallahassee. 

1. Purpose 

The purpose of this study was to investigate a method for 
stimulating 11 the intellectual development of college students which 
takes only a short time to administer and which is largely 
content-free" (p. 234). 

2. Rationale 

Typical ly> fewer than forty percent of college freshmen demonstrate 
the thinking strategies normally associated with formal operational 
reasoning. The demonstration of these strategies is thought to be 
directly related to success and inversely related to anxiety in 
courses which require formal reasoning. Previous studies have shown 
that Inducing cognitive growth in college students is possible, but 
have required the students to enroll in a course to receive the 
necessary experiences. This approach takes an extended period of time 
and the content of the course (e. g. t science) may cause such anxiety 
that the student will not enroll. 

The authors felt that what was needed was a method for inducing 
the necessary cognitive growth through meaningful experiences which 
were: "largely content-free, challenging to the student without 
being overwhelming, and brief" (p. 234). 

The method selected was to have the students play the game of 
Master Mind with an observer. A previous study had demonstrated that 
Master Mind was an effective tool "for teaching deductive reasoning" 
(p. 235). 
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3. Research Design and Procedures 

f 

A pretest of cognitive development (covering combinatorial 
analysis, proportional reasoning, and separation and control of 
variables) selected from materials developed by Renner, Prickett, 
and Renner was used as a pretest with a total of 185 college students 
Test-retest reliability of the instrument was tested with seventy 
students not participating in the experiment and was found to be .88. 

On the pretest, seventy-six students irere found to be concrete 
operational and thirty-nine of these students agreed to participate 
in the study. The students were randomly assigned to one of three 

groups: 

a. a structured interaction (SI) treatment group, 

b. a neutral interaction (NI) treatment group, or 

c. a Hawthorne control group. 

Each student met with an observer one hour per week for four 
weeks. Students in the SI group were asked questions which would 
cause them to reflect upon their problem-solving strategies while 
playing Master Mind with the observer. Students in the NI group 
played Master Mind with the observer, but without help or questioning 
Students in the Hawthorne group played checkers with the observer. 

The same test of cognitive development was administered to each 

i 

student during the fifth week of the experiment. 

4. Fin d ings 

A single factor analysis of covariance was used to compare result 
of the pretest and posttest for the three groups. A statistically 
significant value of F ipC - ,05) resulted in the use of the 
Newman-Keuls test to make palrwise comparisons of adjusted group 
means* The only differences found tg be significant were those 
between the SI group and eafh of the other groups* Seventy-eight 
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percent of the subjects in the SI group changed from concrete 
operational reasoning to either transitional (57*) or formal 
operational (21%) reasoning. 

5. Interpretations 

•* 

The authors conclude: "although playing Master Mind can have a 
positive effect \:he cognitive development of concrete-operational 
college students, problem-related interaction while playing the game 
is essential" (p. 239). The authors provide an analogy between 
science and Master Mind as evidence of the importance of the method. 
They also indicate a need to examine further how Master Mind may be 
played to maximize cognitive growth and to examine the long-term 
effects of the treatment. 

Abstractor's Comments 

The most striking facet of this study is that the authors actually 
did art adequate power analysis prior to beginning the experiment. 
The authors chose to set limits on type I (■*) errors, type II 
errors, and effect size (ES). A power analysis based upon these 
limits indicates that the required sample size is 11. Differences 
which are statistically significant may be of no practical 
significance. By selecting an effect size, the authors indicated how 
big a difference would be "big enough" to be considered practical. 
This study indicates that it may be possible to raise a student's 
level of cognitive development in a short period of time. Since many 
teachers find formal operational thinking to be a necessary 
prerequisite for success in their courses, the method explored by the 
authors may be a practical way to provide success for more students 
in these courses without resorting to time-consuming and expensive 
remedial courses. 
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Several questions remain concerning the use of Master Mind to 
raise a student's level of cognitive development.. The authors 
identify the student's role in playing the game (code maker or code 
breaker) and the long-term effects as factors requiring further 
study. Other factors for study include: 

1. What skills are required of the "observer" to provide the 
most effective interaction?, 

2. What types of questions are the most effective in causing 
students to reflect upon their own problem-solving strategies? 

3. What effect will such training have on student performance 

in college classes? / 

. . * / 

/ 
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